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EXECUTIVE  SUMMARY 


This  study  analyzed  monthly  expenditures  for  eleven  General  Ledger 
Accounts,  and  their  composite  total,  that  were  identified  by  AFLC/FMB 
as  being  "contract  delivery"  related.  The  purpose  of  the  analysis  was 
to  develop  a  method  for  forecasting  monthly  expenditures  so  that  the 
funding  for  those  General  Ledger  Accounts  could  be  better  tracked  and 
controlled.  Of  the  eleven  General  Ledger  Accounts  examined,  a  Pareto 
analysis  showed  that  GLA  500  accounted  for  approximately  99%  of  the 
total  outlays  involved.  Therefore,  this  study  concentrated  on  only 
that  one  General  Ledger  Account.  Moving  averages,  time  series 
analysis,  exponential  smoothing,  data  offsets  to  simulate  leadtimes, 
and  monthly  percentage  of  annual  outlays  were  some  of  the  techniques 
tested  in  developing  the  forecasting  methodology.  Correlation 
matrices  were  run  to  determine  any  significant  relationships. 
Regression  equations  were  calculated  for  those  cases  found  to  be 
significant . 

The  database  used  for  this  study  covered  the  time  period  October  1984 
to  April  1991.  The  monthly  outlays  during  that  time  averaged 
approximately  $124  million  with  a  standard  deviation  of  approximately 
$41  million.  Because  of  the  high  degree  of  scatter  in  the  GLA  500 
data.  Obligations  data  and  Accrued  Expenditures  Unpaid  data  were  also 
analyzed.  Neither  of  these  additional  data  sources  proved  beneficial. 

V  The  two  forecasting  methods  which  were  the  best  were:  "time  series" 
and  "monthly  percentage  of  the  annual  outlay". ^Because  of  the  recent 
decreases  in  the  AFLC  Budget,  the  time  series  forecasting  equation  has 
a  negative  slope  of  $1,515  million  per  month.  This  could  give  mislead¬ 
ing  values  if  forecasts  were  attempted  very  far  into  the  future.  For 
example,  a  monthly  forecast  for  five  years  from  now  would  indicate 
expenditures  approximately  $90  million  less  than  current  levels,  not 
necessarily  a  valid  assumption.  The  forecast  based  on  a  monthly 
percentage  of  annual  outlays  would  be  more  stable  and  although  it  has 
a  marginally  larger  average  residual  error  between  the  forecast  value 
and  the  actual  value,  it  would  probably  be  a  better  method  for  use  by 
FMB. 
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Foreword 


HQ  AFLC/FMB  needs  to  project  outlays/disbursements  on  a  monthly  basis. 
Their  current  forecasting  method  uses  annual  deliveries  for  each 
fiscal  year  as  a  baseline  and  multiplies  that  baseline  by  historical 
monthly  percentages  to  project  monthly  System  Support  Division 
(SSD)  deliveries  across  a  six  year  forecast  horizon.  Due  to  a  variety 
of  reasons,  such  as  "progress  payments"  and  contract  deliveries  being 
earlier  or  later  than  scheduled,  that  method  has  failed  to  provide  the 
accuracy  that  FMB  requires.  FMB  therefore  requested  XPS  to  assist  them 
in  developing  a  better  forecasting  method. 

During  the  past  two  years,  XRII  (formerly  HMIS)  has  conducted  two 
studies  involving  System  Support  Division  deliveries.  The  overall 
findings  were  that  17%  of  contract  deliveries  occur  in  the  month  for 
which  they  are  scheduled?  27%  are  early  and  56%  are  late.  Those 
studies  also  concluded  that  annual  forecasts  had  an  average  error  of 
5.3%  but  that  monthly  forecast  errors  averaged  29.5%. 

Methods  identified  in  this  study  will  be  used  to  forecast  monthly 
disbursements  for  eleven  General  Ledger  Accounts  (which  include 
Accrued  Expenditures  Unpaid  and  Progress  Payments)  to  insure  that 
projected  outlay/disbursement  levels  are  not  exceeded. 
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Objectives 


There  were  two  objectives  addressed  in  this  study.  The  first  objective 
was  to  attempt  to  identify  whether  funding  outlay  trends  exist  in 
monthly  disbursements  for  eleven  General  Ledger  Accounts.  The  second 
objective,  which  flows  from  the  first  objective,  was  to  attempt  to 
identify  the  source  of  monthly  fluctuations  in  disbursements.  The 
accomplishment  of  those  objectives  will  permit  FMB  to  better  track  and 
control  the  General  Ledger  Accounts  involved. 
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Methodology 


During  the  past  two  years,  XRII  conducted  several  studies  involving 
System  Support  System  deliveries.  Copies  of  the  reports  documenting 
those  studies  were  reviewed,  as  was  the  data  that  was  used  in  those 
studies.  Those  studies  found  that  only  17%  of  contract  deliveries 
occur  in  the  month  for  which  they  are  scheduled.  They  also  found  that 
27%  of  the  deliveries  are  earlier  and  56%  are  later  than  the  scheduled 
delivery  date.  In  addition,  contract  progress  payments  are  creating 
obligations,  sometimes  years,  before  the  delivery  of  the  first  item  of 
product.  The  XRII  studies  also  concluded  that  the  method  used  for 
annual  forecasting  was  satisfactory  but  that  monthly  forecasting 
methods  produced  an  average  error  of  29.5%,  which  was  unsatisfactory. 

The  scope  of  this  study  was  limited  to  the  analysis  of  the  following 
eleven  General  Ledger  Accounts  which  were  identified  by  FMB: 

GLA  Description 

430  Discounts  Earned  on  Purchases 

440  Recoveries  For  Retail  Stock  Losses 

445  Material  Return  Credits  Received 

465  Trade-In  Allowances  on  Purchases 

500  Purchases  At  Cost 

540  Transportation  Expenses 

550  Repair  Expense 

59005  Contract  Termination  Expense 
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59010  Renegotiation  Costs/Price  Increases 

59011  Service  Contract  Cost 
59013  Modification  Costs 

The  actual  monthly  expenditures  for  each  GLA  were  obtained  from  the 
Division  Consolidated  Trial  Balance  report  for  the  time  period 
October  1984-April  1991.  The  data  was  analyzed  for  trends  and 
deviation  around  the  mean  value  of  monthly  disbursement.  A  Pareto 
analysis  of  the  disbursements  for  the  eleven  GLAs  indicated  that  one 
GLA,  GLA  500  (Purchases  At  Cost),  accounted  for  approximately  99X  of 
the  composite  total  disbursements.  A  comparison  of  the  composite 
disbursements  and  the  GLA  500  disbursements  is  shown  in  Figure  1. 
Therefore,  this  study  restricted  itself  to  analysis  of  that  single 
GLA.  The  monthly  disbursements  for  GLA  500  are  shown  in  Figure  2.  A 
histogram  of  that  data  is  shown  in  Figure  3.  The  GLA  500  disbursements 
had  an  average  monthly  value  of  $124.3  million  with  a  standard 
deviation  of  $41.3  million.  The  other  ten  GLAs  had  average  monthly 
disbursements  ranging  from  a  high  of  $854  thousand  to  a  low  of  $340. 

The  analysis  performed  on  the  GLA  500  data  centered  on  four  separate 
areas : 

a)  analysis  of  GLA  500  data  by  itself. 

b)  comparison  of  obligation  data  and  GLA  500  data. 

c)  the  effect  of  Accrued  Expenditures  Unpaid. 

d)  use  of  monthly  percentages. 
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Analysis 


Analysis  Of  GLA  500  Data  By  Itself; 

The  historical  disbursement  data  for  GLA  500  was  used  as  a  predictor 
of  future  monthly  outlays.  Lag  periods  of  one  to  twelve  months  were 
used  to  simulate  "seasonal"  effects.  A  correlation  matrix  was 
calculated  to  determine  the  degree  of  relationship  between  GLA  500 
outlays  and  the  various  lag  values.  The  highest  "r"  value  obtained  was 
0.47,  which  does  not  indicate  a  good  statistical  relationship  based  on 
a  value  of  1.0  signifying  a  perfect  relationship  between  two 
variables.  When  the  forecasts  generated  using  these  lags  were  compared 
with  the  actual  values,  the  average  residual  difference  between  actual 
and  forecast  was  in  the  $30  million  to  $40  million  range,  depending  on 
which  lag  period  was  used  (see  Figure  4). 

In  an  attempt  to  flatten  out  the  peaks  and  valleys  in  the  data, 
various  "averaging"  techniques  were  examined.  Moving  averages  of  two 
to  twelve  months  were  calculated,  as  were  single  exponential  smoothed 
values  using  smoothing  factors  of  0.1,  0.2,  0.4,  and  0.6.  The  best 
correlation  coefficient  obtained  using  these  techniques  was  0.58. 
Although  this  value  is  better  than  the  one  obtained  using  the  lag 
method,  it  is  still  a  poor  fit  for  forecasting  purposes.  The  average 
residual  values  between  the  actual  outlays  and  the  forecasts  using 
this  method  were  in  the  mid  to  upper  $20  million  range  (see  Figures  5 
and  6). 
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An  analysis  was  made  combining  the  "lag"  and  the  "averaging" 
techniques.  Moving  averages  and  exponentially  smoothed  forecasts  were 
calculated  for  lag  periods  of  1-12  months.  The  best  correlation 
coefficient  obtained  was  0.58,  the  same  as  for  the  "averaging" 
technique  by  itself.  Combining  the  two  methods  did  not  improve  the 
forecasting  relationship. 

The  final  study  of  the  GLA  500  data  by  itself  was  a  time  series 
analysis.  The  monthly  outlays  were  regressed  against  "time"  and  a 
prediction  equation  (Forecast  =  161.158  -  1.515  x  time)  was 
calculated.  In  the  regression  equation,  the  dollar  values  are 
millions  and  time  is  "months  since  December  1986".  The  time  period 
December  1986-April  1991  was  used  to  calculate  the  coefficients  for 
the  equation.  Data  prior  to  December  1986  was  excluded  from  the 
regression  equation  since  several  large  swings  occurred  in  the  data 
during  that  earlier  period  and  a  slight  downward  trend  began  in  the 
data  around  that  time.  The  correlation  coefficient  for  the  time  series 
analysis  was  -0.64,  giving  an  "r  squared"  for  the  regression  equation 
of  0.41  (approximately  AIX  of  the  variance  in  disbursements  is 
accounted  for  by  the  regression  equation).  This  was  the  best 
correlation  coefficient  obtained  so  far.  .The  monthly  residual 
differences  between  the  actual  outlays  and  those  calculated  by  the 
regression  equation  averaged  $22  million,  also  the  best  obtained  so 
far.  Figure  7  shows  the  regression  line  plotted  with  the  actual  data 
for  December  1986-April  1991.  Figure  8  shows  how  the  regression  line 
would  have  compared  to  the  actual  data  for  the  entire  period 
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October  1984-April  1991.  Because  of  the  negative  slope,  $1,515  million 
per  month,  care  must  be  exercised  in  using  the  regression  equation  for 
forecasting  very  far  into  the  future.  For  example,  a  monthly  forecast 
for  five  years  from  nov  would  be  $90  million  less  than  today's  value. 

Comparison  of  Obligation  Data  And  GLA  500  Data; 

In  an  effort  to  further  explain  the  wide  swings  in  the  monthly 
outlays,  other  sources  of  data  were  researched  for  comparison  with  the 
"outlays"  data  that  had  already  been  analyzed.  Data  was  obtained 
showing  the  "obligations"  for  October  1982-April  1991  (see  Figure  9). 
An  evaluation  was  made  to  see  whether  "obligations"  made  at  one  point 
in  time  showed  up  as  "outlays"  some  leadtime  later.  Lags  of  0-50 
months  were  used  to  simulate  leadtime.  Each  of  the  "lagged" 
obligations  were  compared  with  the  outlays  and  a  correlation  matrix 
was  calculated.  The  best  correlation  coefficient  obtained  was  0.46. 
The  average  residual  differences  between  actual  and  forecast  monthly 
values  using  this  method  ranged  from  $37  million  to  $70  million.  As 
can  be  seen  from  Figure  9,  the  obligations  data  has  severe  peaks  and 
valleys.  To  dampen  the  data,  exponential  smoothing  (using  factors  of 
0.2,  0.4,  and  0.6)  was  used  along  with  a  0-24  month  lag  for  simulated 
leadtime.  The  correlation  coefficients  improved  to  0.62  in  two  of  the 
cases  (0.2  smoothing  of  obligations  with  a  3  month  lag  and  a  6  month 
lag).  However  this  is  not  quite  as  good  as  the  correlation 
coefficient  of  0.64  obtained  with  the  time  series  analysis  discussed 
above.  Figures  10-12  show  the  average  monthly  residual  differences 
associated  with  this  analysis. 
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The  Effect  Of  Accrued  Expenditures  Unpaid; 


A  portion  of  the  GLA  500  type  disbursements  is  broken  out  into  a 
separate  account  OAC  45.  Data  was  obtained  for  OAC  45  (Accrued 
Expenditures  Unpaid)  for  the  period  December  1984-June  1991.  That  data 
is  shown  on  Figure  13  along  with  the  other  GLA  500  data.  As  can  be 
seen,  the  Accrued  Expenditures  Unpaid  fluctuates  both  positive  and 
negative.  This  is  a  revolving  account  that  shows  net  monthly  changes 
and  depending  on  how  much  backlog  exists,  the  net  change  for 
any  given  month  between  what  was  paid  and  what  needs  to  be  paid  can  be 
either  plus  or  minus.  To  provide  a  better  database  the  two  "accounts” 
were  combined  as  shown  on  Figure  14.  It  was  hoped  that  the  overall 
turbulence  of  the  data  would  smooth  out,  but  this  was  not  the  case  as 
can  be  seen  by  comparing  Figures  13  and  14.  Calculating  a  correlation 
matrix  of  the  combined  data  in  Figure  14  versus  obligations  data  with 
simulated  leadtimes  of  0-50  months  resulted  in  correlation 
coefficients  less  than  0.35. 

Use  Of  Monthly  Percentages; 

The  final  analysis  performed,  examined  whether  outlays  changed 

depending  on  which  month  of  the  year  was  involved.  The  current 

forecasting  method  being  used  by  FMB  is  based  on  calculating  the 

percent  of  annual  outlays  which  historically  have  occurred  during  each 

month  of  the  year.  After  the  annual  funding  levels  have  been 

established  via  the  budgeting  system,  those  factors  (percents)  are 
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used  to  forecast  how  the  outlays  will  occur  on  a  monthly  basis.  The 
current  factors,  based  on  an  average  of  FY88-FY90,  are  listed  below: 


Month 

Percent 

October 

0.099 

November 

0.067 

December 

0.100 

January 

0.086 

February 

0.071 

March 

0.096 

April 

0.090 

May 

0.073 

June 

0.094 

July 

0.067 

August 

0.094 

September 

0.062 

An  Analysis  of  Variance  test  was  run  on  the  monthly  data  for  PY85- 
FY90.  The  test  indicated  that  no  statistical  evidence  existed  to 
support  the  hypothesis  that  outlay  percentage  is  affected  by  the  month 
in  which  the  outlay  occurred.  Therefore,  if  "percent  of  annual 
disbursements"  is  to  be  used  as  a  basis  for  forecasting  monthly 
outlays,  a  factor  of  0.083  (one- twelfth)  should  be  used.  Using 
historical  percentages  (the  current  method)  for  October  1984-November 
1986  with  the  GLA  500  data  for  December  1986-April  1991  resulted  in  an 
average  residual  difference  of  $40.8  million  between  actual  and 
forecast  monthly  outlays.  Using  the  0.083  factor  with  the  GLA  500  data 
for  December  1986-April  1991  resulted  in  an  average  residual 
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for  December  1986-April  1991  resulted  in  an  average  residual 
difference  of  $22.6  million  between  the  actual  and  forecast  monthly 
outlays.  This  compares  favorably  with  the  $21.9  million  average 
residual  value  resulting  from  use  of  the  regression  equation  discussed 
above.  Because  of  the  potential  problems  associated  with  the  negative 
slope  of  the  regression  equation,  as  discussed  previously,  the  use  of 
a  0.083  monthly  factor  of  the  annual  budget  will  probably  result  in 
the  best  forecasting  method  for  GLA  500  monthly  disbursements. 


Results 


With  regard  to  the  Objectives  established  at  the  outset  of  this  study: 

a)  No  real  trend,  which  would  assist  in  forecasting,  was  found  to 
exist  in  monthly  disbursements.  However,  recent  historical  data 
did  exhibit  a  decrease  in  outlays  which  conforms  to  the  overall 
decrease  in  the  Defense  Budget.  This  was  discussed  earlier. 

b)  The  main  source  of  monthly  fluctuations  in  disbursements  for 
the  eleven  General  Ledger  Accounts  results  from  GLA  500, 
Purchases  At  Cost,  which  makes  up  approximately  99%  of  the 
composite  disbursements  for  the  eleven  GLAs.  Obligations  data  and 
Accrued  Expenditures  Unpaid  data  were  also  examined  to  ascertain 
the  cause  of  scatter  in  the  GLA  500  data.  Neither  of  those  sources 
provided  an  answer.  Progress  payments  and  erratic  deliveries  (as 
identified  in  several  XRII  studies)  were  the  only  explanations 
found  for  the  volatility  of  the  data.  Another  possible  source  of 
monthly  fluctuations  could  be  the  operating  procedures  at  the 
Finance  Center  in  Denver.  The  Finance  Center  operations  were  not 
addressed  in  detail  in  this  study  since  they  involve  more  than 
"disbursements”  and  impact  more  than  AFLC.  Hence,  to  smooth  out 
the  peaks  and  valleys  in  the  disbursement  workload  would  result  in 
other  workload  being  disrupted  and  other  customers  being  impacted. 
The  Accrued  Expenditures  Unpaid  portion  of  the  Finance  Center 
operation  was  analyzed  since  that  area  directly  impacts  on 
disbursements.  That  analysis  is  documented  in  the  body  of  this 


report. 
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Several  analyses  were  performed  to  develop  a  method  for  forecasting 
monthly  disbursements  for  GLA  500.  The  two  methods  which  were 
determined  to  be  best  are  essentially  comparable  from  an  accuracy 


standpoint.  The  two  methods  were;  time  series  regression,  and  a 
straight-line  monthly  factor  (0.083)  of  annual  projections.  The 
distributions  of  residual  errors  resulting  from  the  two  forecasting 
methods,  plus  the  current  method,  for  the  period  December  1986-April 
1991  are  shown  below; 


Method  Actual  Minus  Forecast  ($  million)  Average 


< 

-40 

-20 

0 

0 

20 

40 

> 

-60 

-60 

-40 

-20 

20 

40 

60 

60 

Current  Method 

9 

3 

7 

7 

7 

9 

2 

8 

40.8 

0.083  Month.  Factor 

1 

3 

10 

11 

16 

7 

3 

1 

22.6 

Regression 

1 

1 

10 

12 

17 

6 

5 

0 

21.9 

In  all  three  cases,  forecasts  are  split  fairly  evenly  between  over 
forecasts  and  under  forecasts.  It  should  also  be  noted  that  in  the 
case  of  the  ”0.083  monthly  factor"  and  "regression"  forecasting 
methods,  even  though  the  average  forecasting  error  is  approximately 
$22  million,  large  errors  are  possible  for  individual  months.  The  two 
methods  had  errors  in  excess  of  $40  million  during  eight  and  seven 
months  respectively  during  the  December  1986-=April  1991  period 
examined.  If  this  frequency  and/or  magnitude  of  error  is  unacceptable, 
FMB  can  add  a  "safety  level"  to  the  forecast  value  to  guard  against 
months  where  the  actual  disbursements  are  higher  than  the  forecast.  Of 
course,  that  "safety  level"  will  increase  the  excess  in  the  months 
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where  the  forecast  is  already  larger  than  the  actual  disbursements. 
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Observations /Recommendations 


This  study  examined  forecasting  methods  based  on:  moving  averages, 
exponential  smoothing,  data  lags  to  simulate  leadtime,  regression, 
time  series  analysis,  and  monthly  factoring  of  annual  disbursements. 
Two  methods  were  judged  to  be  best  based  on  the  average  absolute  value 
of  the  residual  error  between  the  actual  and  forecasted  monthly 
disbursements.  The  method  based  on  time  series  analysis  of  historical 
data  resulted  in  a  negative  slope  of  approximately  $1.5  million  per 
month.  While  this  method  is  acceptable  for  near  term  projections,  care 
must  be  used  as  the  forecast  horizon  is  increased.  The  other  method, 
based  on  assigning  a  factor  to  each  month,  should  not  produce  similar 
adverse  effects  as  the  forecast  horizon  is  increased  since  major 
changes  in  operational  environment  will  have  already  been  captured  in 
the  annual  figures  that  the  monthly  factors  are  applied  against.  The 
normal  cautions  always  exist  regardless  of  what  method  is  used;  namely 
that  as  time  passes  the  assumptions  and  trends  that  existed  in  the 
data  which  generated  the  forecasting  method  could  change  and  require 
modification. 

The  recommendations  from  this  study  are: 

1)  Use  a  monthly  factor  of  0.083  to  allocate  the  annual  disburse¬ 
ment  projection  to  monthly  increments.  Previous  studies  by  XRII 
(formerly  MMIS)  have  indicated  that  the  annual  projections  have 
a  mean  absolute  percent  error  of  5.3%,  a  rate  that  FMB  has 
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indicated  they  can  live  with. 


2)  Give  consideration  to  establishing  a  "safety  level"  to  increase 
the  forecast  sufficiently  to  preclude  unacceptable  cash 
shortfalls  during  those  months  where  the  forecast  is  less  than 
the  actual  disbursement.  This  is  an  area  that  FMB  must  address. 
Although  XPS  can  not  directly  make  a  recommendation  in  this 
area,  if  FMB  is  able  to  quantify  the  degree  of  risk  they  are 
willing  to  take  regarding  a  cash  shortfall,  XPS  can  assist  in 
determining  the  size  of  any  "safety  level"  established.  As 
stated  previously,  the  establishment  of  a  "safety  level"  will 
amplify  the  funding  excess  for  those  months  where  the  forecast 
already  exceeds  the  actual  disbursements. 
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